As cloud based storage systems, online backup services, and large file transfer systems become commonplace, storage and bandwidth requirements are two critical factors which, if controlled, can lead to key competitive advantages. However, there is the competing factor that users of these online storage systems desired that their data be secure from both the operators of such services and other users from the services.
The traditional way of implementing an online file system as a service to customers is to allow each user to upload files to the online storage system through a world wide web interface, across a file transfer protocol connection, or other file transport protocol. Typically, the files are sent to the service unencrypted meaning the operator of the service could access the user's data. In some cases, the services encrypt the data for storage within their service while in other cases the services provide client software that encrypts data prior to uploading the data to the service. In cases where the client encrypts data prior to uploading, the traditional method is to 1) have a client encrypt a file using some unique encryption key (e.g., user password or public key); 2) have the client upload the encrypted file; 3) store the encrypted file on the server.
The traditional means of providing an encrypted upload to clients of online storage systems presents a number of drawbacks. First, it is possible that two distinct clients have exact copies of data (e.g., pictures, movies, songs, documents) that each client stores in the online storage system. However, since both files would be uniquely encrypted by each client, the encrypted data is not identical and the online storage system must store two copies of the same unencrypted data (albeit in encrypted form). The service is unable to store just one copy of this data as each client would only be able to decrypt the copy that client uploaded.
One alternative approach is to have the service manage the encryption keys on the server side and, thus, be capable of decrypting files that the clients upload. The server is then able to determine that two decrypted files are identical and store just one copy of the decrypted data. Then, when each client requests its data the server encrypts the data with that client's encryption key and transmits the requested file. This approach compromises security in that the operator has complete access to each client's data. Unscrupulous operators may rifle through their client's data. Further, the service itself is a single point of weakness and a successful breach in security exposes all of the clients' data.
Under either the traditional or alternative approach described, each time a client saves a file to the online storage system, the client must upload the encrypted data regardless of whether a copy of that file (in encrypted or unencrypted form) already exists in the online storage system. Thus, even under the alternative approach described the client wastes valuable bandwidth resources uploading data the server does not need because the online storage system already has the data in its storage pool.